{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "uxaPxE0ZtXPj"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/generation/openai/chatgpt/plugins/langchain-docs-plugin.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/pinecone-io/examples/blob/master/learn/generation/openai/chatgpt/plugins/langchain-docs-plugin.ipynb)\n",
        "\n",
        "# Building a LangChain Docs Plugin for ChatGPT\n",
        "\n",
        "In this walkthrough we setup a ChatGPT plugin.\n",
        "\n",
        "Before running this notebook you should have already initialized the retrieval API and have it running locally or elsewhere (like on Digital Ocean). More detailed instructions for the setup and deployment can be [found in the video here](https://youtu.be/hpePPqKxNq8).\n",
        "\n",
        "We will summarize the instructions (specific to the Pinecone datastore) before moving on to the walkthrough."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RFqw_Y0ptXPk"
      },
      "source": [
        "## App Quickstart"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ex9RTutztXPk"
      },
      "source": [
        "1. Install Python 3.10 if not already installed.\n",
        "\n",
        "2. Clone the `chatgpt-retrieval-plugin` repository:\n",
        "\n",
        "```\n",
        "git clone git@github.com:openai/chatgpt-retrieval-plugin.git\n",
        "```\n",
        "\n",
        "_**Note**: To see how we setup the *hosted app* on DigitalOcean [refer to this video](https://youtu.be/hpePPqKxNq8), otherwise continue to setup the app locally by following the remaining steps._\n",
        "\n",
        "3. Navigate to the app directory:\n",
        "\n",
        "```\n",
        "cd /path/to/chatgpt-retrieval-plugin\n",
        "```\n",
        "\n",
        "4. Install `poetry`:\n",
        "\n",
        "```\n",
        "pip install poetry\n",
        "```\n",
        "\n",
        "5. Create a new virtual environment:\n",
        "\n",
        "```\n",
        "poetry env use python3.10\n",
        "```\n",
        "\n",
        "6. Install the `retrieval-app` dependencies:\n",
        "\n",
        "```\n",
        "poetry install\n",
        "```\n",
        "\n",
        "7. Set app environment variables:\n",
        "\n",
        "* `BEARER_TOKEN`: Secret token used by the app to authorize incoming requests. We will later include this in the request `headers`. The token can be generated however you prefer, such as using [jwt.io](https://jwt.io/).\n",
        "\n",
        "* `OPENAI_API_KEY`: The OpenAI API key used for generating embeddings with the `text-embedding-ada-002` model. [Get an API key here](https://platform.openai.com/account/api-keys)!\n",
        "\n",
        "8. Set Pinecone-specific environment variables:\n",
        "\n",
        "* `DATASTORE`: set to `pinecone`.\n",
        "\n",
        "* `PINECONE_API_KEY`: Set to your Pinecone API key. This requires a free Pinecone account and can be [found in the Pinecone console](https://app.pinecone.io/).\n",
        "\n",
        "* `PINECONE_ENVIRONMENT`: Set to your Pinecone environment, looks like `us-east1-gcp`, `us-west1-aws`, and can be found next to your API key in the [Pinecone console](https://app.pinecone.io/).\n",
        "\n",
        "* `PINECONE_INDEX`: Set this to your chosen index name. The name you choose is your choice, we just recommend setting it to something descriptive like `\"openai-retrieval-app\"`. *Note that index names are restricted to alphanumeric characters, `\"-\"`, and can contain a maximum of 45 characters.*\n",
        "\n",
        "8. Run the app with:\n",
        "\n",
        "```\n",
        "poetry run start\n",
        "```\n",
        "\n",
        "If running the app locally you should see something like:\n",
        "\n",
        "```\n",
        "INFO:     Uvicorn running on http://0.0.0.0:8000\n",
        "INFO:     Application startup complete.\n",
        "```\n",
        "\n",
        "In that case, the app has automatically connected to our index (specified by `PINECONE_INDEX`), if no index with that name existed beforehand, the app creates one for us.\n",
        "\n",
        "Now we're ready to move on to populating our index with some data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rVsnYnVJtXPl"
      },
      "source": [
        "## Required Libraries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uUrzZ35ztXPl"
      },
      "source": [
        "There are a few Python libraries we must `pip install` for this notebook to run, those are:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3vZ-IYr9tXPl",
        "outputId": "39a6b998-dcce-4dda-a4b0-112c39e47795"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m420.2/420.2 KB\u001b[0m \u001b[31m6.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m39.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m27.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m264.6/264.6 KB\u001b[0m \u001b[31m16.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m158.8/158.8 KB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.2/114.2 KB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.1/49.1 KB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install -qU langchain tiktoken tqdm"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jgZqGDV6tXPm"
      },
      "source": [
        "## Preparing Data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DHZ91rE0tXPm"
      },
      "source": [
        "In this example, we will download the LangChain docs from [langchain.readthedocs.io/](https://langchain.readthedocs.io/latest/en/). We get all `.html` files located on the site like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eEZp2PXztXPm"
      },
      "outputs": [],
      "source": [
        "!wget -r -A.html -P rtdocs https://python.langchain.com/en/latest/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6BRVl45YtXPm"
      },
      "source": [
        "This downloads all HTML into the `rtdocs` directory. Now we can use LangChain itself to process these docs. We do this using the `ReadTheDocsLoader` like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EfhjpaKktXPm",
        "outputId": "1ba0a053-7099-485c-88b4-e2396f77d324"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/Users/jamesbriggs/opt/anaconda3/envs/ml/lib/python3.9/site-packages/langchain/document_loaders/readthedocs.py:30: GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system (\"lxml\"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.\n",
            "\n",
            "The code that caused this warning is on line 30 of the file /Users/jamesbriggs/opt/anaconda3/envs/ml/lib/python3.9/site-packages/langchain/document_loaders/readthedocs.py. To get rid of this warning, pass the additional argument 'features=\"lxml\"' to the BeautifulSoup constructor.\n",
            "\n",
            "  _ = BeautifulSoup(\n",
            "/Users/jamesbriggs/opt/anaconda3/envs/ml/lib/python3.9/site-packages/langchain/document_loaders/readthedocs.py:46: GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system (\"lxml\"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.\n",
            "\n",
            "The code that caused this warning is on line 46 of the file /Users/jamesbriggs/opt/anaconda3/envs/ml/lib/python3.9/site-packages/langchain/document_loaders/readthedocs.py. To get rid of this warning, pass the additional argument 'features=\"lxml\"' to the BeautifulSoup constructor.\n",
            "\n",
            "  soup = BeautifulSoup(data, **self.bs_kwargs)\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "389"
            ]
          },
          "execution_count": 1,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from langchain.document_loaders import ReadTheDocsLoader\n",
        "\n",
        "loader = ReadTheDocsLoader('rtdocs')\n",
        "docs = loader.load()\n",
        "len(docs)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "osJ8_mGitXPm"
      },
      "source": [
        "This leaves us with `389` processed doc pages. Let's take a look at the format each one contains:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Uri6XGGHtXPn",
        "outputId": "ea5d1daf-72c1-4f18-885a-923b55eb4dbc"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Document(page_content='.rst\\n.pdf\\nWelcome to LangChain\\n Contents \\nGetting Started\\nModules\\nUse Cases\\nReference Docs\\nLangChain Ecosystem\\nAdditional Resources\\nWelcome to LangChain#\\nLarge language models (LLMs) are emerging as a transformative technology, enabling\\ndevelopers to build applications that they previously could not.\\nBut using these LLMs in isolation is often not enough to\\ncreate a truly powerful app - the real power comes when you are able to\\ncombine them with other sources of computation or knowledge.\\nThis library is aimed at assisting in the development of those types of applications. Common examples of these types of applications include:\\n❓ Question Answering over specific documents\\nDocumentation\\nEnd-to-end Example: Question Answering over Notion Database\\n💬 Chatbots\\nDocumentation\\nEnd-to-end Example: Chat-LangChain\\n🤖 Agents\\nDocumentation\\nEnd-to-end Example: GPT+WolframAlpha\\nGetting Started#\\nCheckout the below guide for a walkthrough of how to get started using LangChain to create an Language Model application.\\nGetting Started Documentation\\nModules#\\nThere are several main modules that LangChain provides support for.\\nFor each module we provide some examples to get started, how-to guides, reference docs, and conceptual guides.\\nThese modules are, in increasing order of complexity:\\nPrompts: This includes prompt management, prompt optimization, and prompt serialization.\\nLLMs: This includes a generic interface for all LLMs, and common utilities for working with LLMs.\\nDocument Loaders: This includes a standard interface for loading documents, as well as specific integrations to all types of text data sources.\\nUtils: Language models are often more powerful when interacting with other sources of knowledge or computation. This can include Python REPLs, embeddings, search engines, and more. LangChain provides a large collection of common utils to use in your application.\\nChains: Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.\\nIndexes: Language models are often more powerful when combined with your own text data - this module covers best practices for doing exactly that.\\nAgents: Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.\\nMemory: Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.\\nChat: Chat models are a variation on Language Models that expose a different API - rather than working with raw text, they work with messages. LangChain provides a standard interface for working with them and doing all the same things as above.\\nUse Cases#\\nThe above modules can be used in a variety of ways. LangChain also provides guidance and assistance in this. Below are some of the common use cases LangChain supports.\\nAgents: Agents are systems that use a language model to interact with other tools. These can be used to do more grounded question/answering, interact with APIs, or even take actions.\\nChatbots: Since language models are good at producing text, that makes them ideal for creating chatbots.\\nData Augmented Generation: Data Augmented Generation involves specific types of chains that first interact with an external datasource to fetch data to use in the generation step. Examples of this include summarization of long pieces of text and question/answering over specific data sources.\\nQuestion Answering: Answering questions over specific documents, only utilizing the information in those documents to construct an answer. A type of Data Augmented Generation.\\nSummarization: Summarizing longer documents into shorter, more condensed chunks of information. A type of Data Augmented Generation.\\nQuerying Tabular Data: If you want to understand how to use LLMs to query data that is stored in a tabular format (csvs, SQL, dataframes, etc) you should read this page.\\nEvaluation: Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.\\nGenerate similar examples: Generating similar examples to a given input. This is a common use case for many applications, and LangChain provides some prompts/chains for assisting in this.\\nCompare models: Experimenting with different prompts, models, and chains is a big part of developing the best possible application. The ModelLaboratory makes it easy to do so.\\nReference Docs#\\nAll of LangChain’s reference documentation, in one place. Full documentation on all methods, classes, installation methods, and integration setups for LangChain.\\nReference Documentation\\nLangChain Ecosystem#\\nGuides for how other companies/products can be used with LangChain\\nLangChain Ecosystem\\nAdditional Resources#\\nAdditional collection of resources we think may be useful as you develop your application!\\nLangChainHub: The LangChainHub is a place to share and explore other prompts, chains, and agents.\\nGlossary: A glossary of all related terms, papers, methods, etc. Whether implemented in LangChain or not!\\nGallery: A collection of our favorite projects that use LangChain. Useful for finding inspiration or seeing how things were done in other applications.\\nDeployments: A collection of instructions, code snippets, and template repositories for deploying LangChain apps.\\nDiscord: Join us on our Discord to discuss all things LangChain!\\nTracing: A guide on using tracing in LangChain to visualize the execution of chains and agents.\\nProduction Support: As you move your LangChains into production, we’d love to offer more comprehensive support. Please fill out this form and we’ll set up a dedicated support Slack channel.\\nnext\\nQuickstart Guide\\n Contents\\n  \\nGetting Started\\nModules\\nUse Cases\\nReference Docs\\nLangChain Ecosystem\\nAdditional Resources\\nBy Harrison Chase\\n    \\n      © Copyright 2023, Harrison Chase.\\n      \\n  Last updated on Mar 21, 2023.\\n  ', lookup_str='', metadata={'source': 'rtdocs/langchain.readthedocs.io/en/latest/index.html'}, lookup_index=0)"
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "docs[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GVAcsY7JtXPn"
      },
      "source": [
        "We access the plaintext page content like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JPljMKcytXPn",
        "outputId": "47087370-e65f-40f4-d8b9-9df30a897c1b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            ".rst\n",
            ".pdf\n",
            "Welcome to LangChain\n",
            " Contents \n",
            "Getting Started\n",
            "Modules\n",
            "Use Cases\n",
            "Reference Docs\n",
            "LangChain Ecosystem\n",
            "Additional Resources\n",
            "Welcome to LangChain#\n",
            "Large language models (LLMs) are emerging as a transformative technology, enabling\n",
            "developers to build applications that they previously could not.\n",
            "But using these LLMs in isolation is often not enough to\n",
            "create a truly powerful app - the real power comes when you are able to\n",
            "combine them with other sources of computation or knowledge.\n",
            "This library is aimed at assisting in the development of those types of applications. Common examples of these types of applications include:\n",
            "❓ Question Answering over specific documents\n",
            "Documentation\n",
            "End-to-end Example: Question Answering over Notion Database\n",
            "💬 Chatbots\n",
            "Documentation\n",
            "End-to-end Example: Chat-LangChain\n",
            "🤖 Agents\n",
            "Documentation\n",
            "End-to-end Example: GPT+WolframAlpha\n",
            "Getting Started#\n",
            "Checkout the below guide for a walkthrough of how to get started using LangChain to create an Language Model application.\n",
            "Getting Started Documentation\n",
            "Modules#\n",
            "There are several main modules that LangChain provides support for.\n",
            "For each module we provide some examples to get started, how-to guides, reference docs, and conceptual guides.\n",
            "These modules are, in increasing order of complexity:\n",
            "Prompts: This includes prompt management, prompt optimization, and prompt serialization.\n",
            "LLMs: This includes a generic interface for all LLMs, and common utilities for working with LLMs.\n",
            "Document Loaders: This includes a standard interface for loading documents, as well as specific integrations to all types of text data sources.\n",
            "Utils: Language models are often more powerful when interacting with other sources of knowledge or computation. This can include Python REPLs, embeddings, search engines, and more. LangChain provides a large collection of common utils to use in your application.\n",
            "Chains: Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.\n",
            "Indexes: Language models are often more powerful when combined with your own text data - this module covers best practices for doing exactly that.\n",
            "Agents: Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.\n",
            "Memory: Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.\n",
            "Chat: Chat models are a variation on Language Models that expose a different API - rather than working with raw text, they work with messages. LangChain provides a standard interface for working with them and doing all the same things as above.\n",
            "Use Cases#\n",
            "The above modules can be used in a variety of ways. LangChain also provides guidance and assistance in this. Below are some of the common use cases LangChain supports.\n",
            "Agents: Agents are systems that use a language model to interact with other tools. These can be used to do more grounded question/answering, interact with APIs, or even take actions.\n",
            "Chatbots: Since language models are good at producing text, that makes them ideal for creating chatbots.\n",
            "Data Augmented Generation: Data Augmented Generation involves specific types of chains that first interact with an external datasource to fetch data to use in the generation step. Examples of this include summarization of long pieces of text and question/answering over specific data sources.\n",
            "Question Answering: Answering questions over specific documents, only utilizing the information in those documents to construct an answer. A type of Data Augmented Generation.\n",
            "Summarization: Summarizing longer documents into shorter, more condensed chunks of information. A type of Data Augmented Generation.\n",
            "Querying Tabular Data: If you want to understand how to use LLMs to query data that is stored in a tabular format (csvs, SQL, dataframes, etc) you should read this page.\n",
            "Evaluation: Generative models are notoriously hard to evaluate with traditional metrics. One new way of evaluating them is using language models themselves to do the evaluation. LangChain provides some prompts/chains for assisting in this.\n",
            "Generate similar examples: Generating similar examples to a given input. This is a common use case for many applications, and LangChain provides some prompts/chains for assisting in this.\n",
            "Compare models: Experimenting with different prompts, models, and chains is a big part of developing the best possible application. The ModelLaboratory makes it easy to do so.\n",
            "Reference Docs#\n",
            "All of LangChain’s reference documentation, in one place. Full documentation on all methods, classes, installation methods, and integration setups for LangChain.\n",
            "Reference Documentation\n",
            "LangChain Ecosystem#\n",
            "Guides for how other companies/products can be used with LangChain\n",
            "LangChain Ecosystem\n",
            "Additional Resources#\n",
            "Additional collection of resources we think may be useful as you develop your application!\n",
            "LangChainHub: The LangChainHub is a place to share and explore other prompts, chains, and agents.\n",
            "Glossary: A glossary of all related terms, papers, methods, etc. Whether implemented in LangChain or not!\n",
            "Gallery: A collection of our favorite projects that use LangChain. Useful for finding inspiration or seeing how things were done in other applications.\n",
            "Deployments: A collection of instructions, code snippets, and template repositories for deploying LangChain apps.\n",
            "Discord: Join us on our Discord to discuss all things LangChain!\n",
            "Tracing: A guide on using tracing in LangChain to visualize the execution of chains and agents.\n",
            "Production Support: As you move your LangChains into production, we’d love to offer more comprehensive support. Please fill out this form and we’ll set up a dedicated support Slack channel.\n",
            "next\n",
            "Quickstart Guide\n",
            " Contents\n",
            "  \n",
            "Getting Started\n",
            "Modules\n",
            "Use Cases\n",
            "Reference Docs\n",
            "LangChain Ecosystem\n",
            "Additional Resources\n",
            "By Harrison Chase\n",
            "    \n",
            "      © Copyright 2023, Harrison Chase.\n",
            "      \n",
            "  Last updated on Mar 21, 2023.\n",
            "  \n"
          ]
        }
      ],
      "source": [
        "print(docs[0].page_content)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YHz25GKftXPn",
        "outputId": "0c38da33-9ff7-4633-a5d7-fe5bc5e96075"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            ".md\n",
            ".pdf\n",
            "Glossary\n",
            " Contents \n",
            "Chain of Thought Prompting\n",
            "Action Plan Generation\n",
            "ReAct Prompting\n",
            "Self-ask\n",
            "Prompt Chaining\n",
            "Memetic Proxy\n",
            "Self Consistency\n",
            "Inception\n",
            "MemPrompt\n",
            "Glossary#\n",
            "This is a collection of terminology commonly used when developing LLM applications.\n",
            "It contains reference to external papers or sources where the concept was first introduced,\n",
            "as well as to places in LangChain where the concept is used.\n",
            "Chain of Thought Prompting#\n",
            "A prompting technique used to encourage the model to generate a series of intermediate reasoning steps.\n",
            "A less formal way to induce this behavior is to include “Let’s think step-by-step” in the prompt.\n",
            "Resources:\n",
            "Chain-of-Thought Paper\n",
            "Step-by-Step Paper\n",
            "Action Plan Generation#\n",
            "A prompt usage that uses a language model to generate actions to take.\n",
            "The results of these actions can then be fed back into the language model to generate a subsequent action.\n",
            "Resources:\n",
            "WebGPT Paper\n",
            "SayCan Paper\n",
            "ReAct Prompting#\n",
            "A prompting technique that combines Chain-of-Thought prompting with action plan generation.\n",
            "This induces the to model to think about what action to take, then take it.\n",
            "Resources:\n",
            "Paper\n",
            "LangChain Example\n",
            "Self-ask#\n",
            "A prompting method that builds on top of chain-of-thought prompting.\n",
            "In this method, the model explicitly asks itself follow-up questions, which are then answered by an external search engine.\n",
            "Resources:\n",
            "Paper\n",
            "LangChain Example\n",
            "Prompt Chaining#\n",
            "Combining multiple LLM calls together, with the output of one-step being the input to the next.\n",
            "Resources:\n",
            "PromptChainer Paper\n",
            "Language Model Cascades\n",
            "ICE Primer Book\n",
            "Socratic Models\n",
            "Memetic Proxy#\n",
            "Encouraging the LLM to respond in a certain way framing the discussion in a context that the model knows of and that will result in that type of response. For example, as a conversation between a student and a teacher.\n",
            "Resources:\n",
            "Paper\n",
            "Self Consistency#\n",
            "A decoding strategy that samples a diverse set of reasoning paths and then selects the most consistent answer.\n",
            "Is most effective when combined with Chain-of-thought prompting.\n",
            "Resources:\n",
            "Paper\n",
            "Inception#\n",
            "Also called “First Person Instruction”.\n",
            "Encouraging the model to think a certain way by including the start of the model’s response in the prompt.\n",
            "Resources:\n",
            "Example\n",
            "MemPrompt#\n",
            "MemPrompt maintains a memory of errors and user feedback, and uses them to prevent repetition of mistakes.\n",
            "Resources:\n",
            "Paper\n",
            "previous\n",
            "Writer\n",
            "next\n",
            "LangChain Gallery\n",
            " Contents\n",
            "  \n",
            "Chain of Thought Prompting\n",
            "Action Plan Generation\n",
            "ReAct Prompting\n",
            "Self-ask\n",
            "Prompt Chaining\n",
            "Memetic Proxy\n",
            "Self Consistency\n",
            "Inception\n",
            "MemPrompt\n",
            "By Harrison Chase\n",
            "    \n",
            "      © Copyright 2023, Harrison Chase.\n",
            "      \n",
            "  Last updated on Mar 21, 2023.\n",
            "  \n"
          ]
        }
      ],
      "source": [
        "print(docs[5].page_content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iNdtvj_BtXPn"
      },
      "source": [
        "We can also find the source of each document:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iuvpNElntXPn",
        "outputId": "7d0f7ea8-0d02-4c7b-eb8f-b80c6671eb8d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'https://langchain.readthedocs.io/en/latest/glossary.html'"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "docs[5].metadata['source'].replace('rtdocs/', 'https://')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dnsc0SxHtXPn"
      },
      "source": [
        "Looks good, we need to also consider the length of each page with respect to the number of tokens that will reasonably fit within the window of a ChatGPT model. We will use `gpt-3.5-turbo` as the assumed model.\n",
        "\n",
        "### Chunking the Text\n",
        "\n",
        "At the time of writing, `gpt-3.5-turbo` supports a context window of 4096 tokens — that means that input tokens + generated ( / completion) output tokens, cannot total more than 4096 without hitting an error.\n",
        "\n",
        "So we 100% need to keep below this. If we assume a very safe margin of ~2000 tokens for the input prompt into `gpt-3.5-turbo`, leaving ~2000 tokens for conversation history and completion.\n",
        "\n",
        "With this ~2000 token limit we may want to include *five* snippets of relevant information, meaning each snippet can be no more than **400** token long.\n",
        "\n",
        "To create these snippets we use the `RecursiveCharacterTextSplitter` from LangChain. To measure the length of snippets we also need a *length function*. This is a function that consumes text, counts the number of tokens within the text (after tokenization using the `gpt-3.5-turbo` tokenizer), and returns that number. We define it like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "u1-4qAiUtXPn"
      },
      "outputs": [],
      "source": [
        "import tiktoken\n",
        "\n",
        "tokenizer = tiktoken.get_encoding('cl100k_base')\n",
        "\n",
        "# create the length function\n",
        "def tiktoken_len(text):\n",
        "    tokens = tokenizer.encode(\n",
        "        text,\n",
        "        disallowed_special=()\n",
        "    )\n",
        "    return len(tokens)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ig7Py5NVtXPn"
      },
      "source": [
        "Note that for the tokenizer we defined the encoder as `\"cl100k_base\"`. This is a specific tiktoken encoder which is used by `gpt-3.5-turbo`, as well as `gpt-4`, and `text-embedding-ada-002` which are models supported by OpenAI at the time of this writing. Other encoders may be available, but are used with models that are now deprecated by OpenAI.\n",
                "\n",
        "You can find more details in the [Tiktoken `model.py` script](https://github.com/openai/tiktoken/blob/main/tiktoken/model.py), or using `tiktoken.encoding_for_model`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uqtyb_nRtXPn",
        "outputId": "1799f08d-eb88-415a-8eb4-64b8de919649"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<Encoding 'cl100k_base'>"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "tiktoken.encoding_for_model('gpt-3.5-turbo')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nLFRqT4rtXPo"
      },
      "source": [
        "With the length function defined we can initialize our `RecursiveCharacterTextSplitter` object like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "S8JX-JF7tXPo"
      },
      "outputs": [],
      "source": [
        "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
        "\n",
        "text_splitter = RecursiveCharacterTextSplitter(\n",
        "    chunk_size=400,\n",
        "    chunk_overlap=20,  # number of tokens overlap between chunks\n",
        "    length_function=tiktoken_len,\n",
        "    separators=['\\n\\n', '\\n', ' ', '']\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "41PQTB4rtXPo"
      },
      "source": [
        "Then we split the text for a document like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GClNt4lDtXPo",
        "outputId": "0a8b1519-5e5a-487f-9285-a5f567e7b633"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chunks = text_splitter.split_text(docs[5].page_content)\n",
        "len(chunks)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rlL_61N0tXPo",
        "outputId": "930afb11-a10c-4d25-f4a7-2e2f112e5c0c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(346, 247)"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "tiktoken_len(chunks[0]), tiktoken_len(chunks[1])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iVEhDDyatXPo"
      },
      "source": [
        "For `docs[5]` we created `2` chunks of token length `346` and `247`.\n",
        "\n",
        "This is for a single document, we need to do this over all of our documents. While we iterate through the docs to create these chunks we will reformat them into the format required by our API app. This format needs to align to the `/upsert` endpoints required document format, which looks like this:\n",
        "\n",
        "```json\n",
        "[\n",
        "    {\n",
        "        \"id\": \"abc\",\n",
        "        \"text\": \"some important document text\",\n",
        "        \"metadata\": {\n",
        "            \"field1\": \"optional metadata goes here\",\n",
        "            \"field2\": 54\n",
        "        }\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"123\",\n",
        "        \"text\": \"some other important text\",\n",
        "        \"metadata\": {\n",
        "            \"field1\": \"another metadata\",\n",
        "            \"field2\": 71,\n",
        "            \"field3\": \"not all metadatas need the same structure\"\n",
        "        }\n",
        "    }\n",
        "    ...\n",
        "]\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b9iinqjrtXPo"
      },
      "source": [
        "Every document *must* have a `\"text\"` field. The `\"id\"` and `\"metadata\"` fields are optional, however, we will include both.\n",
        "\n",
        "The `\"id\"` will be created based on the URL of the text + it's chunk number."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uaMqBpsStXPo",
        "outputId": "53600d0e-9556-48f0-d0b0-fdc2b7a7fd8b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "https://langchain.readthedocs.io/en/latest/glossary.html\n",
            "d5e95d45160a\n"
          ]
        }
      ],
      "source": [
        "import hashlib\n",
        "m = hashlib.md5()  # this will convert URL into unique ID\n",
        "\n",
        "url = docs[5].metadata['source'].replace('rtdocs/', 'https://')\n",
        "print(url)\n",
        "\n",
        "# convert URL to unique ID\n",
        "m.update(url.encode('utf-8'))\n",
        "uid = m.hexdigest()[:12]\n",
        "print(uid)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xi-dECvftXPo"
      },
      "source": [
        "Then use the `uid` alongside chunk number and actual `url` to create the format needed:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o3BZzCwStXPo",
        "outputId": "990301d3-9fde-441b-a0cc-f94c7869864e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'id': 'd5e95d45160a-0',\n",
              "  'text': '.md\\n.pdf\\nGlossary\\n Contents \\nChain of Thought Prompting\\nAction Plan Generation\\nReAct Prompting\\nSelf-ask\\nPrompt Chaining\\nMemetic Proxy\\nSelf Consistency\\nInception\\nMemPrompt\\nGlossary#\\nThis is a collection of terminology commonly used when developing LLM applications.\\nIt contains reference to external papers or sources where the concept was first introduced,\\nas well as to places in LangChain where the concept is used.\\nChain of Thought Prompting#\\nA prompting technique used to encourage the model to generate a series of intermediate reasoning steps.\\nA less formal way to induce this behavior is to include “Let’s think step-by-step” in the prompt.\\nResources:\\nChain-of-Thought Paper\\nStep-by-Step Paper\\nAction Plan Generation#\\nA prompt usage that uses a language model to generate actions to take.\\nThe results of these actions can then be fed back into the language model to generate a subsequent action.\\nResources:\\nWebGPT Paper\\nSayCan Paper\\nReAct Prompting#\\nA prompting technique that combines Chain-of-Thought prompting with action plan generation.\\nThis induces the to model to think about what action to take, then take it.\\nResources:\\nPaper\\nLangChain Example\\nSelf-ask#\\nA prompting method that builds on top of chain-of-thought prompting.\\nIn this method, the model explicitly asks itself follow-up questions, which are then answered by an external search engine.\\nResources:\\nPaper\\nLangChain Example\\nPrompt Chaining#\\nCombining multiple LLM calls together, with the output of one-step being the input to the next.\\nResources:\\nPromptChainer Paper\\nLanguage Model Cascades\\nICE Primer Book\\nSocratic Models\\nMemetic Proxy#',\n",
              "  'metadata': {'url': 'https://langchain.readthedocs.io/en/latest/glossary.html'}},\n",
              " {'id': 'd5e95d45160a-1',\n",
              "  'text': 'Language Model Cascades\\nICE Primer Book\\nSocratic Models\\nMemetic Proxy#\\nEncouraging the LLM to respond in a certain way framing the discussion in a context that the model knows of and that will result in that type of response. For example, as a conversation between a student and a teacher.\\nResources:\\nPaper\\nSelf Consistency#\\nA decoding strategy that samples a diverse set of reasoning paths and then selects the most consistent answer.\\nIs most effective when combined with Chain-of-thought prompting.\\nResources:\\nPaper\\nInception#\\nAlso called “First Person Instruction”.\\nEncouraging the model to think a certain way by including the start of the model’s response in the prompt.\\nResources:\\nExample\\nMemPrompt#\\nMemPrompt maintains a memory of errors and user feedback, and uses them to prevent repetition of mistakes.\\nResources:\\nPaper\\nprevious\\nWriter\\nnext\\nLangChain Gallery\\n Contents\\n  \\nChain of Thought Prompting\\nAction Plan Generation\\nReAct Prompting\\nSelf-ask\\nPrompt Chaining\\nMemetic Proxy\\nSelf Consistency\\nInception\\nMemPrompt\\nBy Harrison Chase\\n    \\n      © Copyright 2023, Harrison Chase.\\n      \\n  Last updated on Mar 21, 2023.',\n",
              "  'metadata': {'url': 'https://langchain.readthedocs.io/en/latest/glossary.html'}}]"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "data = [\n",
        "    {\n",
        "        'id': f'{uid}-{i}',\n",
        "        'text': chunk,\n",
        "        'metadata': {'url': url}\n",
        "    } for i, chunk in enumerate(chunks)\n",
        "]\n",
        "data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e_cID2cktXPp"
      },
      "source": [
        "Now we repeat the same logic across our full dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "referenced_widgets": [
            "afa77c267fb14505851d3ff0f8bb7f1a"
          ]
        },
        "id": "0uSz_lNwtXPp",
        "outputId": "93985b12-e043-4fad-d39f-800d19a20ace"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "afa77c267fb14505851d3ff0f8bb7f1a",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/389 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "2201"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from tqdm.auto import tqdm\n",
        "\n",
        "documents = []\n",
        "\n",
        "for doc in tqdm(docs):\n",
        "    url = doc.metadata['source'].replace('rtdocs/', 'https://')\n",
        "    m.update(url.encode('utf-8'))\n",
        "    uid = m.hexdigest()[:12]\n",
        "    chunks = text_splitter.split_text(doc.page_content)\n",
        "    for i, chunk in enumerate(chunks):\n",
        "        documents.append({\n",
        "            'id': f'{uid}-{i}',\n",
        "            'text': chunk,\n",
        "            'metadata': {'url': url}\n",
        "        })\n",
        "\n",
        "len(documents)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evwnK4KftXPp"
      },
      "source": [
        "We're now left with `2201` documents in the format required by our API."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dNurZ-CPtXPp"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uSvXqSKTtXPp"
      },
      "source": [
        "#### (Optional) Load Dataset from Hugging Face\n",
        "\n",
        "Rather than running the above scripts to build the dataset, you can load a prepared version from Hugging Face Datasets like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 324,
          "referenced_widgets": [
            "e17fc50a411b4976919929cff7afa329",
            "f226a7168836423c9fa255805a2a961a",
            "ba74b3d3c8cf460598ad4aeb8d0b32f7",
            "b3ac85e446684f3da8fb63dbcba1fa20",
            "64380a778acb473685733f2969b64c1e",
            "771af082b5874ecea8b51759ff5cbb52",
            "274feec9651545b3a692d028c31fcac5",
            "7c0b48d09e3845439b6f17e56324ebae",
            "ae16059db75448ebb78617bbeb4cf739",
            "825e264842644347a526bf5dafea9096",
            "3a663fb464b042cd95173d68762a7d9d",
            "a1cedb6c31104cd48214a8ec838a1616",
            "3cff9800fbee4d73a66d7756c6eff7ab",
            "476958e0b9e94a06a96906cbcd61f2df",
            "6359a65a6d934899b67139e72d5991a6",
            "bacca9fafb3b464aa09f359ca707477f",
            "07f9b96cde584dc99bcc7c0fc95d75e7",
            "d0c9ce854ea84ddd9653e207cd8515c4",
            "f09b7d4613454798b6d1ec2a5fe53e7e",
            "87a5ab1eedc74924847ad33c1898790d",
            "a2a6a319a1024535ba4a45f84fc4ec46",
            "3966e35556e94afd9dbed920ab8bbf85",
            "b6da9e973dd84c708dbf78d82fa0f966",
            "e00bc317b3d140c2ad208b358d593a02",
            "87c3a27ea0f34db1be58be50122cbf77",
            "6b4cd53d7bba4c7f8337e497237eaf30",
            "668123cd76bd4a8c919166d605270afe",
            "ddd67395d74d41a5a0fbd3208f904550",
            "c3a4b108a23f444e847eb6735e7d35ce",
            "90e8fd7e27514c108bc81b39061bf81e",
            "bed6a08879154cb19d5fed25c8abe332",
            "d66b2d8b37944a159371cec81bd1e4a1",
            "7ee774773b2646159734e5d87731c178",
            "184f5290a65649f59a99b649a3fcc8af",
            "2550ea896b3a4b39bcb37b374954850c",
            "b136affd421f4c46aae0c9f1cc6f503f",
            "30ce466cba1f45458cfca60be046afa9",
            "b9d6b9723b6f4e63bc26aba581fe047a",
            "67a601ee00f741d29067bf6640012328",
            "a7f21a216faa48ec83d8cfdb3e059daf",
            "592cb1e2e05a401da82a041a5ca14cf4",
            "d724f8048f684a0ba96811b8d7da3154",
            "833bef95bd7741babb7424ff635d6c92",
            "94789f4dbdb74046940ccc9594cf83ce"
          ]
        },
        "id": "Od4iy48HtXPp",
        "outputId": "8f936706-32f4-4434-cc24-dbb48aaab97b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m469.0/469.0 KB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m212.2/212.2 KB\u001b[0m \u001b[31m19.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m110.5/110.5 KB\u001b[0m \u001b[31m12.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m132.9/132.9 KB\u001b[0m \u001b[31m16.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m199.8/199.8 KB\u001b[0m \u001b[31m21.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading and preparing dataset json/jamescalam--langchain-docs to /root/.cache/huggingface/datasets/jamescalam___json/jamescalam--langchain-docs-bcc23a7c6d742f0e/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51...\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "e17fc50a411b4976919929cff7afa329",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading data files:   0%|          | 0/1 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "a1cedb6c31104cd48214a8ec838a1616",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading data:   0%|          | 0.00/2.76M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "b6da9e973dd84c708dbf78d82fa0f966",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Extracting data files:   0%|          | 0/1 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "184f5290a65649f59a99b649a3fcc8af",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Generating train split: 0 examples [00:00, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Dataset json downloaded and prepared to /root/.cache/huggingface/datasets/jamescalam___json/jamescalam--langchain-docs-bcc23a7c6d742f0e/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51. Subsequent calls will reuse this data.\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "Dataset({\n",
              "    features: ['id', 'text', 'source'],\n",
              "    num_rows: 2212\n",
              "})"
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "!pip install -qU datasets\n",
        "\n",
        "from datasets import load_dataset\n",
        "\n",
        "documents = load_dataset('jamescalam/langchain-docs', split='train')\n",
        "documents"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SleL0T2Ht8hX",
        "outputId": "ee2cd387-518c-4b1b-dc44-aa53e68134bd"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'id': '9c04de564ed3-0',\n",
              " 'text': '.rst\\n.pdf\\nWelcome to LangChain\\n Contents \\nGetting Started\\nModules\\nUse Cases\\nReference Docs\\nLangChain Ecosystem\\nAdditional Resources\\nWelcome to LangChain#\\nLarge language models (LLMs) are emerging as a transformative technology, enabling\\ndevelopers to build applications that they previously could not.\\nBut using these LLMs in isolation is often not enough to\\ncreate a truly powerful app - the real power comes when you are able to\\ncombine them with other sources of computation or knowledge.\\nThis library is aimed at assisting in the development of those types of applications. Common examples of these types of applications include:\\n❓ Question Answering over specific documents\\nDocumentation\\nEnd-to-end Example: Question Answering over Notion Database\\n💬 Chatbots\\nDocumentation\\nEnd-to-end Example: Chat-LangChain\\n🤖 Agents\\nDocumentation\\nEnd-to-end Example: GPT+WolframAlpha\\nGetting Started#\\nCheckout the below guide for a walkthrough of how to get started using LangChain to create an Language Model application.\\nGetting Started Documentation\\nModules#\\nThere are several main modules that LangChain provides support for.\\nFor each module we provide some examples to get started, how-to guides, reference docs, and conceptual guides.\\nThese modules are, in increasing order of complexity:\\nPrompts: This includes prompt management, prompt optimization, and prompt serialization.\\nLLMs: This includes a generic interface for all LLMs, and common utilities for working with LLMs.\\nDocument Loaders: This includes a standard interface for loading documents, as well as specific integrations to all types of text data sources.\\nUtils: Language models are often more powerful when interacting with other sources of knowledge or computation. This can include Python REPLs, embeddings, search engines, and more. LangChain provides a large collection of common utils to use in your application.',\n",
              " 'source': 'https://langchain.readthedocs.io/en/latest/index.html'}"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "documents[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "agMlzHTjtXPp"
      },
      "source": [
        "This needs to be reformated into the format we need for the API:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vawiMdJPtXPp",
        "outputId": "5a6fc680-51a5-45a8-9131-a3302324fa92"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'id': '9c04de564ed3-0',\n",
              " 'text': '.rst\\n.pdf\\nWelcome to LangChain\\n Contents \\nGetting Started\\nModules\\nUse Cases\\nReference Docs\\nLangChain Ecosystem\\nAdditional Resources\\nWelcome to LangChain#\\nLarge language models (LLMs) are emerging as a transformative technology, enabling\\ndevelopers to build applications that they previously could not.\\nBut using these LLMs in isolation is often not enough to\\ncreate a truly powerful app - the real power comes when you are able to\\ncombine them with other sources of computation or knowledge.\\nThis library is aimed at assisting in the development of those types of applications. Common examples of these types of applications include:\\n❓ Question Answering over specific documents\\nDocumentation\\nEnd-to-end Example: Question Answering over Notion Database\\n💬 Chatbots\\nDocumentation\\nEnd-to-end Example: Chat-LangChain\\n🤖 Agents\\nDocumentation\\nEnd-to-end Example: GPT+WolframAlpha\\nGetting Started#\\nCheckout the below guide for a walkthrough of how to get started using LangChain to create an Language Model application.\\nGetting Started Documentation\\nModules#\\nThere are several main modules that LangChain provides support for.\\nFor each module we provide some examples to get started, how-to guides, reference docs, and conceptual guides.\\nThese modules are, in increasing order of complexity:\\nPrompts: This includes prompt management, prompt optimization, and prompt serialization.\\nLLMs: This includes a generic interface for all LLMs, and common utilities for working with LLMs.\\nDocument Loaders: This includes a standard interface for loading documents, as well as specific integrations to all types of text data sources.\\nUtils: Language models are often more powerful when interacting with other sources of knowledge or computation. This can include Python REPLs, embeddings, search engines, and more. LangChain provides a large collection of common utils to use in your application.',\n",
              " 'metadata': {'url': 'https://langchain.readthedocs.io/en/latest/index.html'}}"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "documents = [{\n",
        "    'id': doc['id'],\n",
        "    'text': doc['text'],\n",
        "    'metadata': {'url': doc['source']}\n",
        "} for doc in documents]\n",
        "\n",
        "documents[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PwwW8lpCtXPp"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uJg8SYeUtXPp"
      },
      "source": [
        "### Indexing the Docs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6_2pS_qotXPq"
      },
      "source": [
        "We're now ready to begin indexing (or *upserting*) our `documents`. To make these requests to the retrieval app API, we will need to provide authorization in the form of the `BEARER_TOKEN` we set earlier. We do this below:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pizW_qqptXPq"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "BEARER_TOKEN = os.environ.get(\"BEARER_TOKEN\") or \"BEARER_TOKEN_HERE\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2PWlNca8tXPq"
      },
      "source": [
        "Use the `BEARER_TOKEN` to create our authorization `headers`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DHjvFii7tXPq"
      },
      "outputs": [],
      "source": [
        "headers = {\n",
        "    \"Authorization\": f\"Bearer {BEARER_TOKEN}\"\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rn6DbxuVtXPq"
      },
      "source": [
        "We'll perform the upsert in batches of `batch_size`. Make sure that the `endpoint_url` variable is set to the correct location for your running *retrieval-app* API."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "6ab224fc238f4de2865fb08566df6fea",
            "59aa9111efc44cad8cc4cdb039d8a183",
            "3f563339024a4287bbbb20f85b313f7c",
            "e83fc8770c3d4a9a9e0c2d1198db66d9",
            "22b630a8792742db859dfcd03026d1b0",
            "df8d4737c49347de98add5c2fc426f35",
            "13f11b0a05d7466d9edd84f06e56b687",
            "090ab955bf40499d834929e9bf8da0bb",
            "159419e6e6024fc5891ef97e43dce6a7",
            "97bea5162b9f4ee082f2dd2cd411ce89",
            "6277c6e76ea34522b4b4231194cb5634"
          ]
        },
        "id": "DzlPVM0vtXPq",
        "outputId": "cfb53eaf-d6f9-4aff-9df2-ac7a29efdc68"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6ab224fc238f4de2865fb08566df6fea",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/23 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import requests\n",
        "from requests.adapters import HTTPAdapter, Retry\n",
        "from tqdm.auto import tqdm\n",
        "\n",
        "batch_size = 100\n",
        "endpoint_url = \"http://localhost:8000\"\n",
        "s = requests.Session()\n",
        "\n",
        "# we setup a retry strategy to retry on 5xx errors\n",
        "retries = Retry(\n",
        "    total=5,  # number of retries before raising error\n",
        "    backoff_factor=0.1,\n",
        "    status_forcelist=[500, 502, 503, 504]\n",
        ")\n",
        "s.mount('http://', HTTPAdapter(max_retries=retries))\n",
        "\n",
        "for i in tqdm(range(0, len(documents), batch_size)):\n",
        "    i_end = min(len(documents), i+batch_size)\n",
        "    # make post request that allows up to 5 retries\n",
        "    res = s.post(\n",
        "        f\"{endpoint_url}/upsert\",\n",
        "        headers=headers,\n",
        "        json={\n",
        "            \"documents\": documents[i:i_end]\n",
        "        }\n",
        "    )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dMZiWe2YtXPq"
      },
      "source": [
        "With that our LangChain doc records have all been indexed and we can move on to querying."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B7UXES-itXPq"
      },
      "source": [
        "### Making Queries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VFFdoM6itXPq"
      },
      "source": [
        "To query the datastore all we need to do is pass one or more queries to the `/query` endpoint. We can make a few questions related to LangChain and see if we return relevant info:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wBVguntVtXPq",
        "outputId": "60cf68c6-6b59-470d-f657-1657b8ddb119"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<Response [200]>"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "queries = [\n",
        "    {'query': \"What is the LLMChain in LangChain?\"},\n",
        "    {'query': \"How do I use Pinecone in LangChain?\"},\n",
        "    {'query': \"What is the difference between Knowledge Graph memory and buffer memory for \"+\n",
        "     \"conversational memory?\"}\n",
        "]\n",
        "\n",
        "res = requests.post(\n",
        "    f\"{endpoint_url}/query\",\n",
        "    headers=headers,\n",
        "    json={\n",
        "        'queries': queries\n",
        "    }\n",
        ")\n",
        "res"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TrqLTYuQtXPq"
      },
      "source": [
        "Now we can loop through the responses and see the results returned for each query:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jTFRZ2MbtXPq",
        "outputId": "a46b8f84-e557-4734-afea-80d7560ae655"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "----------------------------------------------------------------------\n",
            "What is the LLMChain in LangChain?\n",
            "\n",
            "0.88: nThese are, in increasing order of complexity:\\n\\nð\\x9f“\\x83 LLMs and Prompts:\\n\\nThis includes prompt management, prompt optimization, generic interface for all LLMs, and common utilities for working with LLMs.\\n\\nð\\x9f”\\x97 Chains:\\n\\nChains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.\n",
            "0.88: Chains: Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications. Indexes: Language models are often more powerful when combined with your own text data - this module covers best practices for doing exactly that. Agents: Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents. Memory: Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.\n",
            "0.87: .rst .pdf Chains Chains# Using an LLM in isolation is fine for some simple applications, but many more complex ones require chaining LLMs - either with each other or with other experts. LangChain provides a standard interface for Chains, as well as some common implementations of chains for ease of use. The following sections of documentation are provided: Getting Started: A getting started guide for chains, to get you up and running quickly. Key Concepts: A conceptual guide going over the various concepts related to chains. How-To Guides: A collection of how-to guides. These highlight how to use various types of chains. Reference: API reference documentation for all Chain classes. previous Vector DB Text Generation next Getting Started By Harrison Chase            © Copyright 2023, Harrison Chase.          Last updated on Mar 22, 2023.\n",
            "----------------------------------------------------------------------\n",
            "\n",
            "\n",
            "----------------------------------------------------------------------\n",
            "How do I use Pinecone in LangChain?\n",
            "\n",
            "0.86: from langchain.vectorstores import Pinecone\n",
            "0.85: .md .pdf Pinecone  Contents  Installation and Setup Wrappers VectorStore Pinecone# This page covers how to use the Pinecone ecosystem within LangChain. It is broken into two parts: installation and setup, and then references to specific Pinecone wrappers. Installation and Setup# Install the Python SDK with pip install pinecone-client Wrappers# VectorStore# There exists a wrapper around Pinecone indexes, allowing you to use it as a vectorstore, whether for semantic search or example selection. To import this vectorstore: from langchain.vectorstores import Pinecone For a more detailed walkthrough of the Pinecone wrapper, see this notebook previous PGVector next PromptLayer  Contents    Installation and Setup Wrappers VectorStore By Harrison Chase            © Copyright 2023, Harrison Chase.          Last updated on Mar 22, 2023.\n",
            "0.81: .rst .pdf LangChain Ecosystem LangChain Ecosystem# Guides for how other companies/products can be used with LangChain AI21 Labs AtlasDB Banana CerebriumAI Chroma Cohere DeepInfra Deep Lake ForefrontAI Google Search Wrapper Google Serper Wrapper GooseAI Graphsignal Hazy Research Helicone Hugging Face Modal NLPCloud OpenAI OpenSearch Petals PGVector Pinecone PromptLayer Qdrant Runhouse SearxNG Search API SerpAPI StochasticAI Unstructured Weights & Biases Weaviate Wolfram Alpha Wrapper Writer previous API References next AI21 Labs By Harrison Chase            © Copyright 2023, Harrison Chase.          Last updated on Mar 22, 2023.\n",
            "----------------------------------------------------------------------\n",
            "\n",
            "\n",
            "----------------------------------------------------------------------\n",
            "What is the difference between Knowledge Graph memory and buffer memory for conversational memory?\n",
            "\n",
            "0.84: .ipynb .pdf Conversation Knowledge Graph Memory  Contents  Conversation Knowledge Graph Memory Using in a chain Conversation Knowledge Graph Memory# This type of memory uses a knowledge graph to recreate memory. Let’s first walk through how to use the utilities from langchain.memory import ConversationKGMemory from langchain.llms import OpenAI llm = OpenAI(temperature=0) memory = ConversationKGMemory(llm=llm) memory.save_context({\"input\": \"say hi to sam\"}, {\"ouput\": \"who is sam\"}) memory.save_context({\"input\": \"sam is a friend\"}, {\"ouput\": \"okay\"}) memory.load_memory_variables({\"input\": 'who is sam'}) {'history': 'On Sam: Sam is friend.'} We can also get the history as a list of messages (this is useful if you are using this with a chat model). memory = ConversationKGMemory(llm=llm, return_messages=True) memory.\n",
            "0.84: .md .pdf Key Concepts  Contents  Memory Conversational Memory Entity Memory Key Concepts# Memory# By default, Chains and Agents are stateless, meaning that they treat each incoming query independently. In some applications (chatbots being a GREAT example) it is highly important to remember previous interactions, both at a short term but also at a long term level. The concept of “Memory” exists to do exactly that. Conversational Memory# One of the simpler forms of memory occurs in chatbots, where they remember previous conversations. There are a few different ways to accomplish this: Buffer: This is just passing in the past N interactions in as context. N can be chosen based on a fixed number, the length of the interactions, or other! Summary: This involves summarizing previous conversations and passing that summary in, instead of the raw dialogue itself.\n",
            "0.84: of the raw dialogue itself. Compared to Buffer, this compresses information: meaning it is more lossy, but also less likely to run into context length limits. Combination: A combination of the above two approaches, where you compute a summary but also pass in some previous interactions directly! Entity Memory# A more complex form of memory is remembering information about specific entities in the conversation. This is a more direct and organized way of remembering information over time. Putting it a more structured form also has the benefit of allowing easy inspection of what is known about specific entities. For a guide on how to use this type of memory, see this notebook. previous Getting Started next How-To Guides  Contents    Memory Conversational Memory Entity Memory By Harrison Chase            © Copyright 2023, Harrison Chase.          Last updated on Mar 22, 2023.\n",
            "----------------------------------------------------------------------\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "for query_result in res.json()['results']:\n",
        "    query = query_result['query']\n",
        "    answers = []\n",
        "    scores = []\n",
        "    for result in query_result['results']:\n",
        "        answers.append(result['text'])\n",
        "        scores.append(round(result['score'], 2))\n",
        "    print(\"-\"*70+\"\\n\"+query+\"\\n\\n\"+\"\\n\".join([f\"{s}: {a}\" for a, s in zip(answers, scores)])+\"\\n\"+\"-\"*70+\"\\n\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RsQCGl0EtXPq"
      },
      "source": [
        "The top results are all relevant as we would have hoped. With that we've finished. The retrieval app API can be shut down, and to save resources the Pinecone index can be deleted within the [Pinecone console](https://app.pinecone.io/)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "ml",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.12"
    },
    "orig_nbformat": 4,
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "07f9b96cde584dc99bcc7c0fc95d75e7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "090ab955bf40499d834929e9bf8da0bb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "13f11b0a05d7466d9edd84f06e56b687": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "159419e6e6024fc5891ef97e43dce6a7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "184f5290a65649f59a99b649a3fcc8af": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2550ea896b3a4b39bcb37b374954850c",
              "IPY_MODEL_b136affd421f4c46aae0c9f1cc6f503f",
              "IPY_MODEL_30ce466cba1f45458cfca60be046afa9"
            ],
            "layout": "IPY_MODEL_b9d6b9723b6f4e63bc26aba581fe047a"
          }
        },
        "22b630a8792742db859dfcd03026d1b0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2550ea896b3a4b39bcb37b374954850c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_67a601ee00f741d29067bf6640012328",
            "placeholder": "​",
            "style": "IPY_MODEL_a7f21a216faa48ec83d8cfdb3e059daf",
            "value": "Generating train split: "
          }
        },
        "274feec9651545b3a692d028c31fcac5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "30ce466cba1f45458cfca60be046afa9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_833bef95bd7741babb7424ff635d6c92",
            "placeholder": "​",
            "style": "IPY_MODEL_94789f4dbdb74046940ccc9594cf83ce",
            "value": " 0/0 [00:00&lt;?, ? examples/s]"
          }
        },
        "3966e35556e94afd9dbed920ab8bbf85": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3a663fb464b042cd95173d68762a7d9d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3cff9800fbee4d73a66d7756c6eff7ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_07f9b96cde584dc99bcc7c0fc95d75e7",
            "placeholder": "​",
            "style": "IPY_MODEL_d0c9ce854ea84ddd9653e207cd8515c4",
            "value": "Downloading data: 100%"
          }
        },
        "3f563339024a4287bbbb20f85b313f7c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_090ab955bf40499d834929e9bf8da0bb",
            "max": 23,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_159419e6e6024fc5891ef97e43dce6a7",
            "value": 23
          }
        },
        "476958e0b9e94a06a96906cbcd61f2df": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f09b7d4613454798b6d1ec2a5fe53e7e",
            "max": 2757737,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_87a5ab1eedc74924847ad33c1898790d",
            "value": 2757737
          }
        },
        "592cb1e2e05a401da82a041a5ca14cf4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "59aa9111efc44cad8cc4cdb039d8a183": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df8d4737c49347de98add5c2fc426f35",
            "placeholder": "​",
            "style": "IPY_MODEL_13f11b0a05d7466d9edd84f06e56b687",
            "value": "100%"
          }
        },
        "6277c6e76ea34522b4b4231194cb5634": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6359a65a6d934899b67139e72d5991a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a2a6a319a1024535ba4a45f84fc4ec46",
            "placeholder": "​",
            "style": "IPY_MODEL_3966e35556e94afd9dbed920ab8bbf85",
            "value": " 2.76M/2.76M [00:00&lt;00:00, 6.27MB/s]"
          }
        },
        "64380a778acb473685733f2969b64c1e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "668123cd76bd4a8c919166d605270afe": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "67a601ee00f741d29067bf6640012328": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6ab224fc238f4de2865fb08566df6fea": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_59aa9111efc44cad8cc4cdb039d8a183",
              "IPY_MODEL_3f563339024a4287bbbb20f85b313f7c",
              "IPY_MODEL_e83fc8770c3d4a9a9e0c2d1198db66d9"
            ],
            "layout": "IPY_MODEL_22b630a8792742db859dfcd03026d1b0"
          }
        },
        "6b4cd53d7bba4c7f8337e497237eaf30": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d66b2d8b37944a159371cec81bd1e4a1",
            "placeholder": "​",
            "style": "IPY_MODEL_7ee774773b2646159734e5d87731c178",
            "value": " 1/1 [00:00&lt;00:00, 35.02it/s]"
          }
        },
        "771af082b5874ecea8b51759ff5cbb52": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7c0b48d09e3845439b6f17e56324ebae": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7ee774773b2646159734e5d87731c178": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "825e264842644347a526bf5dafea9096": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "833bef95bd7741babb7424ff635d6c92": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "87a5ab1eedc74924847ad33c1898790d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "87c3a27ea0f34db1be58be50122cbf77": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_90e8fd7e27514c108bc81b39061bf81e",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_bed6a08879154cb19d5fed25c8abe332",
            "value": 1
          }
        },
        "90e8fd7e27514c108bc81b39061bf81e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "94789f4dbdb74046940ccc9594cf83ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "97bea5162b9f4ee082f2dd2cd411ce89": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a1cedb6c31104cd48214a8ec838a1616": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_3cff9800fbee4d73a66d7756c6eff7ab",
              "IPY_MODEL_476958e0b9e94a06a96906cbcd61f2df",
              "IPY_MODEL_6359a65a6d934899b67139e72d5991a6"
            ],
            "layout": "IPY_MODEL_bacca9fafb3b464aa09f359ca707477f"
          }
        },
        "a2a6a319a1024535ba4a45f84fc4ec46": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a7f21a216faa48ec83d8cfdb3e059daf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ae16059db75448ebb78617bbeb4cf739": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b136affd421f4c46aae0c9f1cc6f503f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "info",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_592cb1e2e05a401da82a041a5ca14cf4",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d724f8048f684a0ba96811b8d7da3154",
            "value": 1
          }
        },
        "b3ac85e446684f3da8fb63dbcba1fa20": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_825e264842644347a526bf5dafea9096",
            "placeholder": "​",
            "style": "IPY_MODEL_3a663fb464b042cd95173d68762a7d9d",
            "value": " 1/1 [00:00&lt;00:00,  2.17it/s]"
          }
        },
        "b6da9e973dd84c708dbf78d82fa0f966": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e00bc317b3d140c2ad208b358d593a02",
              "IPY_MODEL_87c3a27ea0f34db1be58be50122cbf77",
              "IPY_MODEL_6b4cd53d7bba4c7f8337e497237eaf30"
            ],
            "layout": "IPY_MODEL_668123cd76bd4a8c919166d605270afe"
          }
        },
        "b9d6b9723b6f4e63bc26aba581fe047a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "ba74b3d3c8cf460598ad4aeb8d0b32f7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7c0b48d09e3845439b6f17e56324ebae",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ae16059db75448ebb78617bbeb4cf739",
            "value": 1
          }
        },
        "bacca9fafb3b464aa09f359ca707477f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bed6a08879154cb19d5fed25c8abe332": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c3a4b108a23f444e847eb6735e7d35ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d0c9ce854ea84ddd9653e207cd8515c4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d66b2d8b37944a159371cec81bd1e4a1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d724f8048f684a0ba96811b8d7da3154": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ddd67395d74d41a5a0fbd3208f904550": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df8d4737c49347de98add5c2fc426f35": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e00bc317b3d140c2ad208b358d593a02": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ddd67395d74d41a5a0fbd3208f904550",
            "placeholder": "​",
            "style": "IPY_MODEL_c3a4b108a23f444e847eb6735e7d35ce",
            "value": "Extracting data files: 100%"
          }
        },
        "e17fc50a411b4976919929cff7afa329": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f226a7168836423c9fa255805a2a961a",
              "IPY_MODEL_ba74b3d3c8cf460598ad4aeb8d0b32f7",
              "IPY_MODEL_b3ac85e446684f3da8fb63dbcba1fa20"
            ],
            "layout": "IPY_MODEL_64380a778acb473685733f2969b64c1e"
          }
        },
        "e83fc8770c3d4a9a9e0c2d1198db66d9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_97bea5162b9f4ee082f2dd2cd411ce89",
            "placeholder": "​",
            "style": "IPY_MODEL_6277c6e76ea34522b4b4231194cb5634",
            "value": " 23/23 [02:01&lt;00:00,  4.23s/it]"
          }
        },
        "f09b7d4613454798b6d1ec2a5fe53e7e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f226a7168836423c9fa255805a2a961a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_771af082b5874ecea8b51759ff5cbb52",
            "placeholder": "​",
            "style": "IPY_MODEL_274feec9651545b3a692d028c31fcac5",
            "value": "Downloading data files: 100%"
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
